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ABSTRACT 

Methodologies for evaluating/documenting partnership 
programs are analyzed and options for evaluating/documenting OERI's 
Educational Partnerships Program (EPP) are proposed. The EPP supports 
alliances between public schools and/or hiaher education and the 
private sector to encourage excellence m education. This paper 
reflects discussions/recommendations of the EPP Study Group, which 
met in Washington, D.C. in November-December, 1990. A two-part 
discussion is provided , which begins with a brief summary of the 
EPP's purpose and the characteristics of the 18 partnership projects 
funded m mid-1990. The second part discusses the proposed evaluation 
options. Options covered for Program Year One (1990-91) include: 
setting up a systematu tracking system; setting up prograri files; 
keeping contact logs by sites, and conducting structured phone 
interviews. Options covered for Program Year Two (1991-92) include 
various evaluation start-up activities; providing either 
project-based formative evaluations or documentation of project 
implementation; preparing project-tased summative evaluations; and 
preparing a summative evaluation of the overall program. Options 
covered for Program Year Three (1992-93) include 'ongoing evaluation 
design, and site profiles. Formative versus summative approaches, 
quantitative versus qualitative data collection, and advantages 
versus disadvantages of various options for each of the three years 
of the EPP are explored. (RLC) 
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Preface 



The U.S. Department of Education's Office of Educational Research 
and Improvement (OERI) funded the first cycle of four year 
fyjojects under its new Educational Partnerships Program in 
S^plfeember 1990. The Educational Partnerships Program is 
authorized by the Educational Partnerships Act of 1988, Title .VI, 
subtitle A, Chapter 5 of the Omnibus Trade and Competitiveness 
Act of 1988 (Pub. L.100-4ie) (20 U.S.C. 5031-5039). This 
legislation also requires the Secretary to conduct an annual 
evaluation of the grants made under the program and to 
disseminate information relating to the activities assisted. 

In performing the evaluation and dissemination responsibility, 
OERI has attempted to design its activities to build on what 
already is known, and to expand knowledge about the 
characteristics of effective partnerships and how to establish, 
sustain, and evaluate them. In order to establish a baseline for 
future work, assess what already is known about educational 
partnerships, and review potential evaluation design options, 
OERI commissioned the following analyses and syntheses: 

Danzberger, Jacqueline P., "EDUCATIONAL PARTNERSHIPS 
PROGRAM-' ANALYSIS OF PROJECT CHARACTERISTICS," 
December, 1990. 

Education Resources Group, "AN OVERVIEW OF EVALUATION 
RESEARCH ON SELECTED EDUCATIONAL PARTNERSHIPS," 
January, 1991. 

Education Resources Group, Inc. "OPTIONS FOR EVALUATING THE 
EDUCATIONAL PARTNERSHIPS PROGRAM," January, 1991. 

Grobe, Terry and Susan P. Curnan and Alan Melchior, 
"SYNTHESIS OF EXISTING KNOWLEDGE AND PRACTICE IN THE FIELD 
OF EDUCATIONAL PARTNERSHIPS," December, 1990. 

This is one of the four commissioned reports. All four are 
available through ERIC. 



INTRODUCTION 



Education Resources Group (ERG) was contracted by OERI to 
prepare an analytic review of methodologies for evaluating and 
documenting partnership programs and to propose options for 
evaluating and documenting the Educational Partnerships Program* 
Within this broad scope of work, ERG was asked to discuss 
particular issues (e.g., partnership "settings" and student 
outcomes) and to delineate formative versus summative approaches, 
quantitative versus qualitative data collection, and the pros and 
cons of the various options. This document reflects discussions 
and recommendations of the Educational Partnerships Program Study 
Group, which met in November and December, 1990. 

The evaluation will serve at least three key audiences, 
including (1) the Legislature, which authorized The Educational 
Partnerships Program through the Educational Partnerships Act of 
1988, (2) interested parties planning to become involved in 
educational partnerships, and (3) the general public. The 
evaluation should look at how the partnership affects outcomes in 
each site, as well as the extent and significance of outcomes 
themselves. These purposes indicate the need for both process 
and outcomes studies. 

The two-part discussion which follows begins with a brief 
summary of the purposes of the Educational Partnerships Program 
and the characteristics of the 18 projects funded in mid*1990. 
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Understanding how the projects differ and are similar will help 
the reader assess the various evaluation options being proposed, 
and potential limitations of the evaluation in general. 

The second part is a discussion of proposed evaluation 
options. Each option will be described, and assertions will be 
made about the relative usefulness, benefits, and disadvantages 
of pursuing them. Within each option, only broad parameters are 
suggested, enough to give the reader a working understanding of 
how the option could be carried out. It is not the purpose here 
to specify a final evaluation plan. 

The partnerships evaluation will begin formally as the 
initial 18 projects go into their second year of operation (fall, 
1991) • OERI encouraged projects to keep records and track 
activities when staff from all projects met as a group in 
Washington, D.C. in December, 1990. xliese activities will be 
useful for evaluators as they begin work this fall. They will 
also be critical for the projects as they design, plan, and carry 
out their own evaluations, which are required by OERI. 

THE EDUCATIONAL PARTNERSHIPS PROGRAM 

The goal of the Educational Partnerships Program is to 
support alliances between public schools and/or higher education 
and the private sector "to encourage excellence in education." 



Alliances can be developed for the following purposes: 

(1) to apply the resources of the private and nonprofit 
sectors of the community to the needs of the elementary 
and secondary schools or the institutions of higher 
education in that community to encourage excellence in 
education; 

(2) to encourage businesses to work with educationally 
disadvantaged students and with gifted students; 



(4) to enrich the career awareness of secondary or 

postsecondary school students and provide exposures to 
the work of the private sector. 

Under these general purposes, partnerships might focus on: 

« 

statewide policies and initiatives; staff training; exposure to 
the work world of businesses^ agencies, and other organizations; 
mentoring, tutoring, and/or internships; and various forms of 
voluntarism. 

Selected Projects 

OERI received 325 partnership proposals, which were reviewed 
by 100 panelists in August, 1990. Eighteen projects around the 
country were awarded first-year grants ranging from $62,000 to 
$325,000; over $3.6 million were awarded. 



Most of the projects are establishing new alliances, 
although several are setting up an "umbrella" structure to 
'Coordinate existing alliances. One project is an extant 



(3) 



to applyXthe resources of communities for the 
improvement of elementary and secondary education or 
higher education; and 
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statewide alliance. Highlights of characteristics of the 18 



projects are listed here. 

•Half the projects will be carried out at the school level, 
including one that is statewide and one that is regional in 
scope • 

•Levels of schooling addressed by the project are mixed. 
For example, while five are at the secondary school level, 
f&ur cover K to 12. 

•The variety of partners is rich: five projects include the 
private, university, community, and parent sectors, four 
involve businesses primarily, and the remaining half include 
one or two of these sectors. 

•Eight projects target disadvantaged, at-risk students, and 
four will work with both at-risk and gifted students. Six 
will servw "alx" students in the school, district or state. 

•Half the projects will focus on career awareness/work 
experience activities. Ten plan some form of staff 
development. Eleven will include tutoring, mentoring, 
internships, and/or volunteer work. (These were key program 
purposes or recommended activities.) 

•About a third are focusing on math/science and/or technology 
as a theme. 



Project-Based Evaluations 



OERI*s request for proposal required an evaluation design to 
be included in project plans. These designs were in general very 
basic; many of the projects did not even name the e valuator. Key * 
features of the evaluation designs are summarized in Table 1. 



Three projects proposed to secure an "external" evaluator; 
two others actually named third-party evaluators. six projects 
have either a project partner or the organization adniinistering 
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Table 1. Project Evaluation Plans 



# of Projects 

Evaluator 

1. Evaluator not named 7 

2. Implementor as evaluator 3 

3. Partmer as evaluator 3 

4. ••External" evaluator to be named 3 

5. Third-party evaluator named 2 

18 

Evaluation Focus 

1. Program inqplementat ion/ student outcomes 8 

2. Program implementation only 6 

3. Student outcomes only 3 

4. Teacher outcomes 1 

18 

Other Not able Characteristics 

1. Evaluation of overall partnership development 6 

2. No listing of possible ••outcome" measures 8 

3. Using a student comparison group 3 



ERLC 



5 

6 



the grant evaluating the program. Seven projects did not name an 
evaluator in their proposals, although many of these have 
selected one by this time. 

f 

The proposed project-based evaluations in general focus on 
the program being implemented (6), student outcomes (3), or both 
(8). One project plans to evaluate teacher outcomes. Three plan 
to use student comparison groups. 

Implications for Evaluation 

Projects' partnership focuses are in general more homogenous 
across the board than might have been expected. Enough variation 
exists, however, to make problematic the implementation of an 
all-project summative evaluation. Individual project objectives 
and targeted groups differ, for example, in emphasis on students 
versus staff, the development of the alliance versus changes in 
student behavior, and reform in school buildings versus district- 
level or statewide reforms. These variations limit the 
evaluator 's opportunity to study across-the-board outcomes for 
students, staff, or partners. They suggest project-by-project 
assessments, at least for specific project activities. These 
could be done by the projects themselves, with technical 
assistance and monitoring by the evaluator. They could 
alternatively be done by the evaluator, who would essentially be 
conducting 18 individual evaluations. 



To study the more general development of partnerships and 
implementation of partnership activities, a systematic 
documentation/ process evaluation can be designed. The evaluator 
might choose to 1) study each project, 2) group projects by 
common focuses (e.g., "statewide" initiatives, gifted students, 
math/ science theme) and then select one or two projects in each 
group for in-depth study, or 3) randomly select six to nine 
projects for intensive documentation (wj^ich includes a series of 
project visits as a framework for data gathering) and designate 
the remaining projects as non-intensive (meaning they would be 
subject to a subset of data collection activities conducted at 
intensive projects) . Variations on this across-the-board 
assessment would affect formative versus summative approaches, 
quantitative and qualitative data collection, and the relative 
costs of each approach. 

OERI expects the projects to receive feedback from the 
evaluator during the course of project implementation. This 
points to formative evaluation or documentation, both of which 
are designed to help projects make mid-course corrections, revise 
objectives, and ultimately improve project outcomes. 

These issues and approaches to evaluating the Educational 
Partnerships Program are discussed more fully on the following 
pages . 
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EVALUATION OPTIONS FOR EDUCATIONAL PARTNERSHIPS PROGRAM 



Program Year One. 1990-91 

No formal evaluation will be conducted for this program in 
its first year: a contract will be awarded in time for 
evaluation to begin with year two. This means the evaluator must 
pick up on 18, year-old projects and begin with a handful of new 
projects OERI hopes to fund in 1991-92. 

It also means that projects must be asked to begin 
systematic record keeping, and where possible, to set up data 
bases for outcomes measures. OERI has asked projects to set up 
such iy stems as soon as possible to ensure routine, in-house data 
collection. This can happen in a number of ways. 

1. Once outcomes are operationally defined (e.g., 

retention rates as a measure of dropout status, choice 
of high school courses as a measure of career 
awareness) , data shouib be kept on groups for which 
behavior changes are expected. Because the 18 projects 
vary In focus on outcomes — ranging from changes in 
statewide policy to changes in student math and science 
scores — a list of outcome variables should be 
developed for each participating group. For example, 
projects focusing on student outcomes should routinely 
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keep student/staff files with such variables as program 
entry/exit, attendance, relevant test scores, credits 
earned, hours of work experience, etc. In doing this, 
projects will be "tracking" project participants' 
behavior over time. 

Projects should also set up and maintain files which 
contain the project proposal, correspondence, activity 
information, meeting agendas and minutes, and other 
relevant documents. This will assist the contracted 
evaluator and the project in constructing/understanding 
project history. It will provide the framework for an 
"institutional memory" about the project. 

Contact logs should be kept by key partners and project 
staff. These can be simple, one-page forms on which 
decisions and important communications are recorded for 
program files. Copies of the form can be kept near the 
phone and completed by hand as the contact is 
occurring. These logs become records in themselves of 
program implementation, and can be used to recall, 
clarify, and order program decisions/events. 

* * OPTION * * OPTION * * 

At least once during 1991, structured phone interviews 
could be conducted with the director and key partners 



of each project. This will provide a snapshot of 
implementation status, start-up issues, concerns, 
obstacles, decisions, modifications, etc. OERI staff 
may want to develop a list of interview questions and 
conduct the interviews themselves, or let a small 
contract to an external group to develop the interview 
form, conduct interviews (probably three or four, 45- 
minute phone interviews per project) , and write them up 
for program files. If this is not possible, the 
contracted evaluator might do this task as an initial 
step in ^jetting up-to-speed on project activities next 
fall. 

Program Year Two. 1991-92 

Evaluation Start-up Activities 

A number of start-up tasks will facilitate the evaluation 
effort. The evaluator must take inventory of partnership 
progress and accomplishments during year one, the status of 
record keeping and program files, key modifications in project 
goals and objectives, staffing patterns, etc. (Project 
continuation proposals to OERI will include a first-year report, 
which the evaluator might use as a framework for this inventory.) 
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* * OPTION * * OPTION ** 



These start-up activities might include an initial or 
follow-up round of phone interviews (costing between $6,500 and 
$8,500) or a round of visits to each project to meet staff and 
review program files and data bases in-person* 

The long-term benefits of at least one round of project 
visits are clear. Project staff meet a real person who is 
conducting an evaluation. The evaluator meets more partnership 
participants than he/she might encounter at annual all-project 
gatherings. The evaluator can see first-hand project files, 
record keeping systems, and data bases. A meeting can be used to 
assist project staff in clarifying goals and objectives, refining 
their own evaluation designs, and understanding the scope, 
requirements, and expectations of the larger evaluation. Some of 
these tasks could be accomplished over the phone or through 
exchange of written materials, but the project visit can solidly 
set the stage for formative evaluation or documentation, and 
summativo evaluation. 

The evaluator would systematically take into account each 
project's goals and objectives, proposed year*two activities, 
status of record keeping and data base systems, and strengths and 
weaknesses in implementation. A subset of projects might be 
visited instead; although this would reduce costs, it would be 
difficult to decide which projects, would not get visits. 
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* * OPTION * * OPTION ** 



Project -Based Formative Evaluations 

One approach to evaluating the partnerships would be to 
provide a formative evaluation for each project. The purpose of 
this would be to collect data at strategic points during program 
implementation, analyze them, and feed back information to 
project staff so they can make mid-course corrections, 
modifications in program plans, and adjustments in level and use 
of resources. 

Two benefits of this approach are that 1) findings can help 
place summative results in context, and 2) OERI can be apprised 
of problems while there is still time to address them. Formative 
evaluation is a very useful and efficient way to study a program 
when it is not i>ossible to visit the program regularly. A 
variety of data collection instruments can be developed or 
adapted to assist the evaluator in efficiently gathering data 
••long-distance. 

Formative evaluation of individual projects can focus on 
program implementation process and/or specific objectives and 
activities in each. The evaluator and project leaders can 
jointly identify key program components and events (e.g., a cycle 
of career awareness /work experience activities, teacher training 

12 



in use of multi-media technology, and partnership decision making 
strategies) « develop operational definitions, performance 
indicators, outcomes benchmar':^ etc., and decide on data 
collection methods to learn about the component or event. Data 
collection methods might include interviews, surveys, criterion- 
referenced or achievement tests, attitudinal scales, participant 
feedback after key events, observations, and participant-focused, 
program-provided data. 

For example, a few projects plan to provide technical 
assistance to ^chools participating in partnership-sponsored 
activities. Technical assistance must be defined, records of 
service provision kept, types of services described, and user 
reactions to services solicited. The evaluator could analyze the 
usefulness of technical assistance and provide focused feedback 
about this at strategic points during the project. This would 
alert the project to its needs and reactions to technical 
assistance; it would also provide recommendations to OERI for 
provision and use of technical assistance services for the other 
projects. 

Conducting formative evaluation for each project can be 
time consuming. Done properly, the task involves customized data 
collection at strategic points for each project. A variety of 
data collection instruments — such as attitude and knowledge 
scales, surveys, and feedback forms — would have to be developed 
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and administered. Data collection would result in written or 
verbal feedback to the projects, which takes time to prepare. If 
project staff do not have analytic capability, the ©valuator may 
have to assist in statistical analyses as well. 



. A basic formative evaluation might include the following set 
of activities for any given project: 

•distribution of one or two questionnaires to project 
partners during the year — questions can be focused on key 
project objectives and activities, the partnership process, 
and perceptions of program quality and efficacy; 

•at least three rounds of structured phone Interviews with 
several key project partners to get updates on project 
activities; 

•collection and analysis of attitudinal and knowledge 
assessments completed by students, teachers, and parents, 
who are the primary recipients of project services in most 
projects; 

•when possible, observations of key project events, such as a 
partnership meeting, a training session, or a career 
awareness activity; 

•a review of program materials recording project activities, 
decisions, and participation patterns; 

•review of outcomes data as they are collected throughout the 
year; and 

•verbal or written feedback mid-year and at the end of the 
school year. 



The cost of such an effort depends qp 1) the evaluator's 
ability to use standardized data collection forms (e.g., for 
phone interviews), and 2) degree of record-keeping sophistication 
at the project level. 
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* * OPINION * * OPTION ** 

Documentation of Program Implementation 

An alternative to formative evaluation, which focuses on 
specific project components, is documentation, which focuses on 
project implementation. Studying how a program is implemented 
serves several purposes. First, it indicates whether or not the 
program is actually pursuing stated goals and objectives. 
Second, it suggests how implementation processes — e.g., 
activities, interactions, and decisions — are affecting 
accomplishments and changing project direction. Third, it helps 
to build an institutional memory so program developers and 
implementors learn from successes and failures. 

The docvunentation process is dynamic and evolutionary. It 
provides for continuous data gathering (in contrast to pre-post 
measures of change found in a more traditional formative 
evaluation), a systematic, focused reporting process, data 
analysis, and feedback. It requires a series of visits to 
designated projects, information gathering focused on key program 
objectives, and preparation of written reports on standardized 
forms that can later be assimilated into a profile of what occurs 
in one project, or an analysis of a particular theme or program 
component across all projects. 
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Project visits can be supplemented with periodic structured 
phone interviews, a survey, perhaps some affective scales or 
content -specific assessments, and review of relevant documents. 
Documenters come together several times a year to debrief, share 
perceptions about the various projects, and work with other team 
members to corroborate findings. Information gathered is coded 
based on pre-developed categories of relevant project processes, 
then entered onto a computerized, qualitative data base for 
future manipulation and analysis. 

The identification of key focuses for documentation is a 
critical component of the process. For the partnership project, 
these might include the organizational structure of the 
partnership, its setting and/or purview, technical assistance, a 
look at other educational improvement programs at the project, 
level of participation by parents, teachers, and students, local 
cost-sharing (a requirement of the grant) , and other factors 
which can be documented in each project. 

Reporting forms are structured so the documenter is forced 
to summarize the status of each focus after every project visit 
or interview. Typically, unique project activities can be 
captured under existing categories on the reporting form, but u 
set of additional items could be generated for each project. 
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A good way to approach the documentation study would be to 
group projects in specific categories — e.g., geographic focus, 
career awareness/work experience focus, subject area focus — and 
select a set of "intensive" projects to be visited five or six 
times over a year. The remaining "non- intensive" projects would 
participate in all other data gathering activities. 

* * OPTION * * OPTION * * 

Project-Based Summative Evaluations 

OERI might go in one of two directions to measure the 
effectiveness of each partnership project. The less costly 
approach would be to rely on the projects* individual evaluation 
studies, some of which promise to be quite credible. A basic 
assumption is that each project will be conducting an evaluation 
and therefore — in some form — maintaining records and 
participant files. The evaluator*s role would include four key 
activities: 

1. clarify with each project its objectives and how they 
will be measured; 

2. provide technical assistance in the design of each 
project's evaluation study and data collection methods; 

3. monitor the projects as they establish record keeping 
systems and data bases, first to ensure consistency and 
appropriateness, second to ensure integrity; and 

4. assist in interpretation of results. 
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This would be a very interactive task, which would require 
one or two rounds of site visits each year by evaluation experts 
would could also provide technical assistance to school sites. 
It woqld be useful because it develops project staffs* capacity 
to do evaluation and relies more on the knowledge of people with 
direct experience in the project. The primary expenses would 
involve the evaluator's time and costs of ongoing communications. 

/ 

A variation on this would be to arrange for group meetings 
of individual project evaluators two or three times per year. 
The benefit of meeting is that they can share with and learn from 
each other, as well as the OERI -contracted evaluator. 

* * OPTION * * OPTION * * 

A more labor intensive approach to individual s\pmative 
evaluations would involve the evaluator collecting outcomes data 
from each project each year, setting up a longitudinal data base 
for each, conducting data analyses, and preparing evaluation 
reports. The evaluator would need to conduct activities one 
(clarifying objectives) and three (monitoring record keeping) 
above, and in addition do the following: 

• agree on outcome measures to be used; 

0 develop instruments to measure outcomes; 

• collect outcomes data from each project; and 

• analyze and interpret results. 

18 



Such data collection and analysis is time consuming because 
individual data files must be checked for accuracy and missing 
information and new data must be added to existing files. If all 
18 projects are included in this outcomes study, two full-time 
data analysts would be needed to establish, maintain, and 
manipulate the 18 data bases. In the end, there will be 18 
different evaluations. 

If OERI were to choose a selection of projects — perhaps 
six to eight — for summative evaluations, the cost could be 
less. This approach would have to be carefully thought out, so 
that there is a viable rationale for focusing on particular 
projects: the most straightforward would be to cluster projects 
by theme (e.g., career awc^reness) or focus (e.g., statewide 
versus school site) . 

* * OPTION * * OPTION * * 
Summative Evaluation of the Overall Program 

Given the broad overall goal of the Educational Partnerships 
Program to establish public/private alliances "to encourage 
excellence in education," it would be difficult to design an 
outcomes evaluation for the project as a whole. Before this 
could be done, agreement would be necessary on an operational 
definition for "excellence in education," the desired goal of the 
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project. One might also argue that "effective partnerships" is 
an outcome that can be measured: again, agreement would be 
necessary on an operational definition of effectiveness for these 
alliances. In effect, this would result in more specific across- 
the-board project objectives. 

What would be necessary steps before a summative evaluation 
could be designed and carried out? Two major issues^ are evident. 

The first is establishing operational definitions for 
"excellence in education" and/or "effective partnerships," which 
would indicate specific program objectives to which all projects 
could aspire. For example, if OERI decides that increased 
student achievement on standardized tests is a valid indicator of 
educational excellence, then all projects would have to relate 
their proposed activities (e.g., career awareness, internships, 
training in technology) directly to academic skills. 

The second problem is identifying measures with which to 
evaluate these objectives across all projects. For example, if 
"effective partnerships" are measured in one way by partner 
satisfaction with program outcomes, all projects would have to 
agree to complete inventories measuring partner satisfaction. 
The end result of such negotiations would be a set of measures 
which could be taken in all projects, then analyzed across projects 
to determine changes in behavior resulting from the program. 
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One way to facilitate an overall program evaluation is to 
set up a computerized data base, into which information from each 
project could be entered. The data base could be used by the 
evaluator and by projects, who could key information into the 
central computer and get access to data whenever they require 
aggregated information about their site(s) . Gathering and 
entering these data — and hiring a data analyst to maintain and 
manipulate the data base ~ can be quite costly. 

A necessary component in an outcomes study of this type is a 
comparison group. The task of identifying a good number of non- 
project partnerships willing to participate in data collection 
activities is formidable, could be frustrating, and is expensive. 
One solution to this is to develop intra-project comparison 
groups, once projects are categorized into smaller groups. 

The partnership projects can be categorized in a number of 

ft 

ways, depending on DERI priorities. Those being implemented 
statewide (3), for example, could be evaluated as a group. Those 
working at the middle school level only (4) could be evaluated as 
a group. Those focusing on gifted children (4) could be 
evaluated as a group. Those focusing on career awareness/ Jobs 
(9) or on math/science (5) could be categorized in groups. Their 
control groups could be students in the other project 
partnerships which are pot focusing in these areas. For example, 
middle school students in the five projects focusing at the 
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secondary level could be comparison students for the four 
projects focusing on the middle school level. 

This type of complicated design is doable. Evaluators in 
New York University's Health Research Program are conducting an 
outcomes study with thell research design characteristics for the 
New Jersey Department of Health. Forty hospitals are 
participating in a demonstration project designed to alleviate 
the nursing shortage in New Jersey. Since the hospitals are 
implel^nting various strategies to accomplish this goal, the 
evaluators have grouped the h9spitals into one of four strategic 
approaches: case management; task restructuring; computerized 
record keeping; or shared governance/decision making. Through a 
factorial design, these groups of hospitals are acting both as 
experimental projects for one set of analyses and as control 
projects for other sets of analyses. Evaluators are also 
conducting a process implemcsntat ion study, which includes two 
project visits to each of the 40 hospitals. Findings from the 
process evaluation will assist researchers in interpreting 
summative evaluation results. 

Plans to conduct an overall summative evaluation of the 
Educational Partnerships Program, perhaps modeled after the 
nursing study which avoids an outside comparison group, could be 
formulated once OERI made a decis%bn about priorities and 
resources. Such a design would clearly be one way to study 



differences in such variables as partnership "setting" (rural 
versus urban versus suburban, statewide versus district level 
versus school-based, etc.) and organizational structure (e.g., 
university administered versus business doi^nated) « Further 
discussion, however, may lead to the conclusion that the basic 
elements necessary for such a design are not in place due to the 
variety of differences and geographical locations of current 
project projects. 

Program Year Three. 1992-93 

Evaluation activities developed for this proqram can be 
refined and maintained during following years. Obvious 
adjustments would include 1) including a handful of new projects 
if OERI funds them, and 2) adding new program data to 
longitudinal data bases, if the contracted evaluator is asked to 
conduct individual summative evaluations or an overall summative 
evaluation. The third program year would also be a good time to 
develop "profiles" of a small selection of projects. Such 
profiles could provide "up-close" looks at selected sites which 
exemplify the best/SBfct interesting partnership initiatives. 
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* * OPTION * * OPTION * * 



Project Profiles 

The te^m **profile** is used here because the traditional 
"case study" approach icay be too labor intensive and costly to 
conduct for the partnership program. Constructing a project 
profile allows more flexibility in terms of data collection and 
content. Documentation can greatly enrich the information 
gathered for a profile, thus resulting in less time on site while 
the profile is being developed. 

OERI must decide if rich descriptions of developing 
partnerships, their activities, and outcomes over a three-year 
period will enhance an understanding of partnerships and their 
role in educational improvement. A decision to go ahead with a - 
series of project profiles must be followed, probably by the end 
of the second project year, with a selection of projects to be 
profiled in year three. 

A professional writer should be identified to work with the 
evaluators in establishing the purpose and content of each 
profile, and in deciding how much more information must be 
gathered during project visits. AW:hough it is not necessary for 
the writer to conduct actual project visits, a day or two on 
project will provide a personal perspective on local color. 



politics, personalities, and impact of the project. 

Very good professional writers aro capable of assimilating a 
large amount of information about a project, conducting a "quick 
study," reading "between the lines," so to speak, and writing 
sharply. They would, of course, have to rely on evaluators to 
analyze and categorize data, provide specific examples to back up 
assertions and conclusions, and ensure "depth" in the profile. 
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A SUMMARY OF OPTIONS 
Program Year One. 1990-91 

Tracking, program files, contact logs by sites 
Structured phone interviews 

Program Y ear Two. 1991-92 

• Evaluation start-up activities 

-Strucvared phone interviews 

OR 

-Round of site visits 

• 

• Project-based formative evaluations 

• Documentation of Project Implementation 

• Project-based summative evaluations 

-Conducted by sites, supervised by evaluator 

OR 

-Conducted by evaluator 

• Summative Evaluation of the Overall Program 
Program Y ear Three, 1992-93 

• Ongoing Evaluation Design 

• Site Profiles 
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